DOCUMENT RESUME 



ED 069 909 



VT 017 666 



TITLE 

INSTITUTION 

PUB DATE 
NOTE 



Composite Evaluation Report for Occupational 
Education in the State of Illinois, Fiscal Year 
1972. 

Illinois State Board of Vocational Education and 
Rehabilitation, Springfield. Vocational and Technical 
Education Div. 

[72] 

50p. 



EDRS PRICE 
DESCRIPTORS 



I DENT IFIERS 



MF-J0.65 HC-$3.29 

Educational Accountability; ’•‘Program Evaluation; 
Program Improvement; *State Programs; Technical 
Education; ^Vocational Development; ‘^Vocational 
Education 
* Illinois 



ABSTRACT 

To promote and assist the development of quality 
state and local occupational education programs, and to help assure 
accountability of federal and stats funds allocated to local 
programs, the Illinois Division of Vocational and Technical Education 
launched a 3-phase evaluation study of its existing system. 
Visitations were made to 71 Local Education Agencies including 
comprehensive high school districts, area vocational centers, and 
community colleges. Evaluations were made with respect to the total 
program's local setting, resources, and limitations* This report is 
organized around these eight major areas of concern: (1) 
administrative organization, (2) personnel, (3) objectives, (4) 
evaluation, (5) occupational programs, (6) resources utilized, (7) 
guidance services, and (8) students served. Conclusions, 
recommendations, and suggested actions are presented separately for 
each area. An appendix containing data samples is included. . (SN) 
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PREFACE 



This report presents the status of occupational education in 
the State of Illinois with recommendations and suggested actions for 
improvement, and is directed toward Local Education Agencies, State 
Education Agencies, and universities. The report was composed by 
twenty-six individuals who were responsible for leading the seventy-one 
evaluation teams during the 1971-72 school year in consortium with 
the Program Approval and Evaluation Unit staff of the Illinois Division 
of Vocational and Technical Education. The team leaders assembled for 
three days to synthesize a massive collection of data, as well as per- 
sonal evaluation experiences, into this final product. The data which 
was utilized included results of questionnaires administered to over 
six thousand students and approximately two thousand local district 
personnel, information from Local District One and Five Year Plans 
for Vocational and Technical Education, school and community data 
forms, and questionnaires completed by team leaders. 

The efforts of the team leaders, culminating in this report, were 
to provide a document useful to the many agencies striving to improve 
occupational education in the State of Illinois. 
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INTRODUCTION 



A system designed to evaluate occupational programs throughout 
Illinois was initiated by the Division of Vocational and Technical 
Education during the 1971-72 school year. The evaluation system, entitled, 
"The Three Phase System for Statewide Evaluation of Occupational Education 
Programs," had three major goals: 

1) To promote and assist with the development of quality 
occupational education programs at the local level. 

2) To provide the Division of Vocational and Technical 
Education with data upon which better state-wide 
planning of occupational education programs can be 
based . 

3) To help assure accountability of Federal and State 
funds allocated to local occupational education pro- 
grams . 

During this first year, visitations were made to 71 Local Education 
Agencies (LEA's). These included comprehensive high school districts, 

Area Vocational Centers, and Community Colleges. On-site evaluation’ teams 
were made up of occupational instructors, business and industrial repre- 
sentatives, students, local district administrators, and university per- 
sonnel. A total of approximately five hundred individuals served as team 
members in these 71 evaluations. 

■ 

Each on-site evaluation focused on a district’s total occupational 
education program, evaluated with respect to its local setting, local 
resources, and local limitations. Programs were not compared from district 
to district, but were evaluated in terms of their own potential and capa- 
city for improvement in their given situation. To guide the teams in pro- 
curing evaluative information, the important factors contributing to the 
success of occupational programs were categorized into eight separate areas 
of concern: Administrative Organization, Personnel, Objectives, Evaluation 
Occupational Programs, Resources Utilized, Guidance Services, and Students 

Served. The visitation interviews, discussions and reports focused on these 
areas. 




The status report presented herein, directed toward local edueation 
agencies, State Education Agencies (DVTE and OSPI) and universities, is orga- 
nized around the eight areas of concern; with Conclusions, Recommendations 
and Suggested Actions presented for each area. This final report was pre- 
pared by the twenty-six individuals who were responsible for leading the 
71 evaluation teams during the 1971-72 school year. These team leaders 
assembled in a workshop setting for three days, with the major task of 
synthesizing local evaluation reports, summaries of collected data, and 
their personal experiences as evaluators into a report which could be utilized 
in future planning by the previously mentioned agencies. 
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GENERAL CONCLUSIONS 



The Vocational Education Act of 1963., and the subsequent Amendments 
of 1968, provided direction for needed changes in occupational offerings. 

The major change entailed de-emphasis of traditional program areas and 
increased emphasis on student needs, interests, and abilities. Occupa- 
tional educators in the State of Illinois are to be commended for 
embracing the 1968 Amendments and for making major strides in implementing 
needed change. 

Since local education agencies have been made responsible for planning 
programs and services, impressive improvements in occupational programs 
have been seen. Moreover, local autonomy has involved a variety of school 
and community personnel in planning, implementing, and evaluating local 
programs. 

Changes visible throughout the state since the passage of the 1968 
Amendments include expansion of program offerings for students, revision 
and improvement of existing programs, mushrooming enrollment in elementary 
occupational information programs, and expanded secondary and post-secondary 
enrollments in occupational programs. Also seen were additions of occupa- 
tional staff, now allotted both the position and the time for thoughtful 
administration of locally directed occupational programs. Increased use 
of arpa vocational centers and joint agreements between districts now 
provide expanded program opportunities to students. These and other 
over-all changes were revealed in the evaluation of local districts during 
the 1971-72 school year. 



Eight Areas of Concern 

More specific conclusions were drawn from the 71 districts evaluated 
during the 1971-72 school year, as they pertained to the eight areas of 
concern dealt with during the on-site evaluations. 

Administrative Organization . There appears to be a high level of 
support for occupational education by local governing boards. The re- 
structuring of local agencies to facilitate program planning and adminis- 
tration, the employment stability of administrative personnel, substantial 
increases in programs offered, and increases in numbers of students served 
are a few examples which made evident the widespread support for occupa- 
tional education. 




Personnel. Ratings of professional personnel throughout the state 

high for unrtr avnari an~a 1 ~ -• . . . . 
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Objectives . Emphasis oh development and utilization of measurable 
objectives has increased vastly over the past two years. The State Agency's 
emphasis on measurable objectives has stimulated similar emphasis in local 
education agencies. Although many local education agencies have yet to 
establish clearly stated, concrete program objectives., over 75% of the 
districts evaluated reported that objectives exist fo\ ;,iost courses. Still, 
the development of measurable objectives for both total and specific 
programs must be expanded and coordinated in local districts throughout the 
state. 

Evaluation . Like objectives, evaluations in local education agencies 
have been spurred by the State Agency. Evaluation was the area most criti- 
cized in districts evaluated during the 1971-72 school year. However, 
there is widespread concern for evaluation and willingness to conduct 
regular local evaluations. The State Agency should offer leadership in the 
planning and direction of local evaluation systems, the development of mate- 
rials, and the coordination of in-service training. 

Occupational Programs . From 1969 to 1972, occupational offerings in 
Illinois have greatly increased. However, state and local agencies have 
failed to adequately forecast and interpret manpower, technological and 
population trends for program planning. Utilizing this information, both 
local and state agencies should be willing to implement new programs, modify 
existing programs, and eliminate ineffective programs. 

Resources Utilized . The 1971-72 evaluations identified a correlation 
between effective advisory committees and quality occupational programs. 
Fifty-nine percent of tho schools evaluated have one advisory committee for 
all occupational areas. Most schools use in-school and out-of-school 
resources wisely to conduct occupational programs, exploiting a wide range 
of possibilities. 

Guidance Services . The pre- service and in-service orientation of 
guidance personnel to occupational counseling is less than adequate. Students 
and occupational teachers, recognizing this deficiency, gave low ratinga to 
counselors' knowledge of the world of work. Like objectives and evaluation, 
guidance services should be singled out for improvement. 

Students Served . Most schools evaluated during the 1971-72 school year 
were rated quite high in serving the needs of individual students. This and 
the massive increase in student enrollment speak well for occupational 
educators' commitment to honor the intent of the 1968 Amendments. Evaluations 
also revealed that over 90% of evaluated schools have one or more clubs for 
occupational students. Clubs have been a vital part of occupational education 
for decades and should be fostered in the future. 
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Statewide Evaluation 



A multi-faceted program such as the occupational education program in 
Illinois must be coordinated and articulated. The Three Phase Evaluation 
System is a useful tool for coordinating planning, evaluation, and setting 
of priorities for occupational education. An equally important function 
of the evaluation system is the gathering and coordinating of data regarding 
strengths and weaknesses of the statewide planning system. Ultimately, the 
goal of the evaluation system is the improvement of occupational education. 
Achievement of this goal requires open and frank assessment of program 
characteristics, identification of needed improvements, and careful esta- 
blishment of priorities for allocating resources. 

The vast changes which have occurred since the 1968 Amendments indicate 
that educators throughout the state are committed to fulfilling the occupa- 
tional education needs of the citizens of Illinois. No single group or 
agency can satisfy present and future needs. A concerted effort must be 
made by many public agencies— the Illinois Division of Vocational and Techni- 
cal Education, the Board of Higher Education, the Office of the Superintendent 
of Public Instruction, local education agencies, and others. These agencies 
must continually strive to meet the need -for viable occupational programs. 

The Division of Vocational and Technical Education and local agency 
personnel are to be commended for sincere involvement in evaluation, complete 
openness, and willingness to examine programs to improve occupational educa- 
tion in the state of Illinois. The remaining pages of this report present 
conclusions, recommendations, and suggested actions for each of the eieht 
areas of concern. 6 
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CONCLUSIONS, RECOMMENDATIONS, AND SUGGESTED ACTIONS 

The following pages present the conclusions of the 1971-72 evaluations 
Recommendations for improvement of occupational education in Illinois are 
based on evaluations of approximately fifteen percent of Illinois local 
educational agencies which offer occupational programs. Findings and 
Recommendations have been categorized according to the eight areas of con- 
cern, with a rating chart preceding each section. A composite of all 
LEA's, this rating chart or histogram is based upon comparisons of each 
local agency's program to its own local potential for serving individuals. 
The degree to which that potential was being achieved was rated by the 
evaluation teams as either excellent, above average, average, below average 
or poor. The histograms presented herein are included to give the reader 
an overview of the strongest and weakest areas of concern in schools 
thrvWghV-ut the state of Illinois. 



Suggested actions within each section of the report have been directed 
to four audiences, although the actions may have implications for other 
audiences. The following legend clarifies abbreviations which are used to 
address suggested actions to a specific audience. 



LEA- All local educational agencies in Illinois 
DVTE- Division of Vocational and Technical Education 
OSPI- Office of the Superintendent of Public Instruction 
U- Universities offering pre-service and in-service 
programs for occupational education personnel 



Administrative Organization 



41 % 




Poor Below Average Above Excellent 

Average Average 



Percentage of evaluated districts within each rating category. Ratings based upon 
the achievement of the district's program compared with the team's perception of 
their local potential for achievement. 
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CONCLUSIONS AND RECOMMENDATIONS 
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Personnel 



Poor 




Below 

Average 



66% 




Average Above Excellent 

Average 



Percentage of evaluated districts within each rating category. Ratings based upon 
the achievement of the district's program compared with the team's perception of 
their local potential for achievement. 
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CONCLUSIONS AND RECOMMENDATIONS 




Objectives 



41 % 




Poor Below Average Above 

Average Average 



Excellent 



Percentage of evaluated districts within each rating category. Ratings based upon 
the achievement of the district's program compared with the team's perception of 
their local potential for achievement. 
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Evaluation 



40 % 




Average 



9 % 




Above 

Average 




t 



Excellent 



Percentage of evaluated districts within each rating category. Ratings based upon 
the achievement of the district's program compared with the team's perception of 
their local potential for achievement 
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c. (LEA) Disseminate findings of 
evaluations and discuss with all 
groups concerned. 
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Occupational Programs 




Percentage of evaluated districts within each rating category 
die achievement of the district's program compared with the 
their local potential for achievement. 



Ratings based upon 
team's perception of 
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b. (LEA,DVTE) An ongoing system of 
follow-up procedures and activities 
should be expanded and given conti- 
nued support by DVTE staff. 
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Resources Utilized 



Poor 



18 % 



Below 

Average 



53 % 




Average Above Excellent 

Average 



Percentage of evaluated districts within each rating category. Ratings based upon 
the achievement of the district's program compared with the team's perception 0 f 
their local potential for achievement. 
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Guidance Services 



40 % 



26 % 




Average Average 

Percentage of evaluated districts within each rating category. Ratings based upon 
the achievement of the district's program compared with the team's perception of 
their local potential for achievement. 
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Students Served 



50 % 




Average Average 



Percentage of evaluated districts within each rating category. Ratings based upon 
the achievement of the district's program compared with the team's perception of 
their local potential for achievement 
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APPENDIX A 



Selected Examples of Data Collected 
During the 1971-72 School Year 



The personal experiences of the team leaders were combined with 
data from five sources to compose this report. Data was tabulated and 
summarized from Local District One and Five Year Plans for the 
seventy-one evaluated agencies, demographic data questionnaires which 
were completed by local administrators prior to the evaluations, 
questionnaires completed by team leaders prior to leaving the evaluation 
site, preliminary evaluation instruments completed by 6,390 occupational 
students, 1,327 teachers and coordinators, 276 counselors, and 242 admi- 
nistrators, and the evaluation reports of the seventy-one districts. 

The following items with their actual results for the 1971-72 school 
year are presented to give an overview of the type of data collected. 

The items are nothing more than examples and are by no means the complete 
data summary. 




Administrative Organization 



1. Rating of local board support < 


of occupational 


education as 


perceived by staff, (staff questionnaire) 








Total 


Tchr . 


Couns . 


Admin. 


High 


.39 


.38 


.39 


.46 


Average 


.52 


.52 


.52 


.48 


Low 


.09 


.10 


.09 


.06 


Number responding: 


1819 


1310 


269 


240 


2. Rate the working relationship between occupational 


instructors and 


administrators in charge of occupational programs. 


(staff quest.) 




Total 


Tchr. 


Couns . 


Admin. 


High 


.39 


.36 


.41 


.50 


Average 


.52 


.54 


.53 


.45 


Low 


.09 


.10 


.06 


.05 


Number responding: 


1843 


1327 


276 


240 


Personnel 










1. Your highest earned degree is 


(staff quest.) 








Total 


Tchr. 


Couns . 


Admin. 


Less than baccalaureate. .04 


.06 


.00 


.00 


Baccalaureate 


.33 


.45 


.03 


.01 


Masters 


.54 


.45 


.82 


.73 


Advanced Certificate 


.07 


.04 


.14 


.16 


Doctorate 


.02 


.01 


.01 


.10 


Number responding: 


1872 


1353 


278 


241 




45 



Personnel, cont'd. 



2. How many years of business or industrial experience 


have you 


had? (staff questionnaire) 








Total 


Tchr. 


Couns . 


Admin . 


None .13 


.10 


.17 


.26 


Less than one year . .06 


.06 


.09 


.06 


1-2 years . 17 


.19 


.14 


.11 


2-3 years .16 


.17 


.15 


.15 


More .48 


.49 


.45 


.46 


Number responding: 1835 


1337 


272 


220 


3. Rate your instructor’s knowledge of required 


1 skills 


for those 


occupations within the area he/she 


is teaching: (student quest.) 


Total 








High .62 








Average . 28 








Below- average .04 








I don't know .06 








Number responding: 6390 








Objectives 








1- Does a set of common objectives exist which 


applies 


to all occu- 


pational programs? (staff questionnaire) 






Total 


Tchr. 


Couns . 


Admin. 


Yes .56 


.54 


.56 


.67 


No .09 


.08 


.09 


.12 


Not sure .35 


.38 


.35 


.21 


Number responding: 1846 


1328 


278 


240 


2. Were you involved in planning and/< 


ar making 


changes 


in your local 


district plan? (staff questionnaire) 






Total 


Tchr. 


Couns. 


Admin . 


Yes .48 


.48 


.30 


.68 


No .52 


.52 


.70 


.32 


Number responding: 1842 


1329 


272 


241 



Eva luation 
1 



Classify your local district's system for evaluation of the total 
occupational program, (staff questionnaire) 



Formal 

Informal 



Number responding: 



Total 


Tchr. 


Couns . 


Admin. 


.39 


.39 


.36 


.40 


.61 


.61 


.64 


.60 


1763 


1261 


266 


236 





Occupational Programs 

1. Rate the course you are taking in terms of preparation for an occu- 
pation: (student questionnaire) 





Total 


High 


.49 


Average 


.43 


Low 


.08 


Number responding: 


6367 



Resources Utilized 

1. What type of advisory committee do you have? (demographic data) 

Total 

One for all areas .59 

Separate one for each .38 

Number responding: 51 (districts) 

Guidance Services 

1. Rate the counselors' awareness of occupational courses and program 
offerings: (staff questionnaire) 





Total 


Tchr. 


Cduns . 


Admin 


High 


.40 


.28 


.78 


.59 


Average 


.46 


.54 


.22 


.35 


Low 


.14 


.18 


.00 


.06 


Number responding: 


1835 


1320 


275 


240 



2. Rate the information you have received from counselors with regard 
to your future occupation: (student questionnaire) 





Total 


High 


.20 


Average 


.46 


Low 


.34 


Number responding: 


6219 



Students Served 

1. I am a member of an occupational student organization such as FFA, 
VICA, FHA, etc. (student questionnaire) 

Total 



.19 

.81 

Number responding: 6101 



O 

ERIC 




Yes 

No 



